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ABSTRACT 

This study is an analysis of the National Assessment 
of Educational Progress (NAEP) School Questionnaire administered 
across three grade levels (grades 4, 8, and 12) during the 1987-88 
school year. The data obtained from the school questionnaire 
represent one level of NAEP's data collection activities, in addition 
to the teacher questionnaires and the various student assessments. 
The analysis was based on an investigation of the relationships 
between policy changes and school climate, measured both by a 
composite scaled score and by indivicxal questionnaire items, and 
individual grouping variables defining school poverty, school size, 
community type, and ethnicity. The results show a fairly corsistent 
pattern of relationships among policy changes and school climate both 
in terms of both how schools ranked on the various grouping 
dimensions and the results of analysis of variance and chi-square 
statistical tests. In terms of policy change, reported changes 
occurred more frequently in large, urban disadvantaged schools wit^ 
predominantly non-white populations. Similarly, the reporting of 
school-related problems (such as absenteeism and tardiness) 
comprising the school climate score tended to be more prevalent in 
bigger city schools containing large minority and less advantaged 
populations. The results sug ^st that reform is occurring in those 
schools where it is needed most. Four data tables are included. 
(Author/TJH) 
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Abstract 

This study is an analysis of the NAEP School Questionnaire administered across three 
grade levels (4, 8 and 12) during the 1987-1988 school year. The data obtained from the 
school questionnaires represent one level of NAEP's data collection activities, in addition to the 
teacher questionnaires and the various student assessments. The analysis was based on an 
investigation of the relationships between policy changes and school climate, measured both 
by a composite scaled score and individual questionnaire items, and individual grouping 
variables defining school poverty, school size, community type and ethnicity. The results 
showed a fairly consistent pattern of relationships among policy changes and school climate 
both in terms of both how schools ranked on the various grouping dimensions and the results 
of Analysis of Variance and Chi Square statistical tests. In terms of policy change, reported 
changes tended to occur more frequently in large, urban disadvantaged schools with 
predominantly nonwhite populations. Similarly, the reporting of school-related problems such 
as absenteeism and iardlness comprising the school climate score tended to be more prevalent 
in bigger city schools containing large minority and less advantaged populations. These results 
suggest that reform is taking place in those schools where it is most needed. 

Background 

The National Assessment of Educational Progress (NAEP) is a federally mandated 
comprehensive national survey of educational achievement spanning three age cohorts of 
American students at the elementary, junior high and high school levels. ETS has been 
conducting this survey on a national basis since 1983 with support from the U.S. Department 
of Education and the National Center for Educational Statistics. In addition to this broad- 
based assessment of achievement, NAEP also collects data on a regular basis in the form of 
student, teacher and school questionnaires. 
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The purpose of collecting this additional data is to tielp NAEP bring its information 
gathering capabilities to boar on important policy questions. As outlined in the prospectus fOr 
a redesigned NAEP. A New Design for a New Era. (Messiclt. Beaton and Lord. 1983). the 
assessment is "practitioner-oriented" in the attempt to link relevant background and program 
variables to educational policy questions and practice. This particular study represents an 
attempt to capitalize on the policy dimensions of NAEP's data collection activities through an 
analysis of policy change and school climate variables at the school building level. 

Introduction 

This analysis takes place within the context of assessing the impact of school reform 
Initiatives. During the past decade, there has been mounting awareness of and interest in 
the implementation of refonn efforts on the part of the federal government down to the local 
school level. Government officials, teachers' groups and individual parents all have voiced 
their concerns over the need to develop policies and practices to counteract the problems 
facing our nations' schools today such as the dropout rate, drug abuse ana declining 
educational perfonnance, to name a few. 

As educational refonn has continued to capture the attention of policy makers, questions 
related to its implementation and impact have become more prominent. As Timar and Kirp 
(1988) note, "little is known about the offects of various reform strategies or the problems 
associated with their implementation" (p. 76). Specifically of interest is the need to assess how 
much reform is taking place. Are state-mandated reform efforts yielding benefits in terms of 
schools actually implementing specific practices? As e jrollaiy. the question of where reform 
is taking place becomes extremely relevant. Is it taking place in those schools where It is most 
needed? Or. is reform mostly an affluent suburban phenomenon, practiced most by those 
able to afford it? Finally, the questions of what types of reforms are taking place and which 
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ones are occurring most frequently are of salience in assessing the extent of specific reform 
Initiatives. 

This analysis will attempt to provide answers to these questions by documenting both 
the nature and extent of policy changes and problems at the school building level. In addition, 
an analysis of policy change and school climate along various dimensions such as urbanicity, 
school size and ethnidty will hopefully yield insights into the questions of the focus and 
relevance of policy reform efforts. 

Instruments 

The data for this study were obtained from the 1987-88 School Characteristics and 
Policies Questionnaire, administered to participating schools in the national assessment. These 
questionnaires were generally filled out by the school principal or another person familiar with 
Infomiation concerning enrollment, facilities, curriculum and staff development. In the majority 
of questionnaires actually returned at the three grade levels, either the principal or vice- 
principal completed the items. Although these questionnaires are directed at the grade levels 
four, eight and twelve, the school-level nature of the questions precludes any direct conclusions 
to be made conceming a particular grade level. Moreover, one should keep in mind the 
question of reliability in interpreting the reported results. Different schools may respond to the 
same questionnaire item with an entirely different perspective, which could possibly affect the 
results. There is, however, no reason to assume such perceptions differ systematically with 
respect to the characteristics on which the variables are reported. 

The first set of questions analyzed concerned policy changes brought about in the 
school since the start of the 1984-1985 school year. The person filling out the questionnaire 
was requested to indicate either changes made by school policy, changes made by state or 
district policy, or no changes effected. There were eight questions in total that were common 
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CO aii three questionnaires at the three grade levels, dealing with policy changes in the following 
areas: 

1 . Lengthening the school day 

2. Lengthening the school year 

3. Establishing a policy of increased homework 

4. Implementing competency testing for promotion or graduation 

5. Establishing new consistently enforced codes of student conduct 

6. Establishing a stricter attendance policy 

7. Establishing grade requirements for participation in athletics or extracurricular 
activities 

8- Implementing any type of perfomiance-based compensation system for teachers, 

such as merit pay, a career ladder, or mentor teacher program 

The second set of questions used in tho analysis concerned the area of school climate, 

whereby the respondent was asked to indicate the degree to which the following matters were 

a problem in the school: 

1 . Student tardiness 

2. Student absenteeism 

3. Student cutting of classes 

4. Physical conflicts among students 

5. Robbery or theft 

6. Vandalism cf school property 

7. Student use of alcohol 

8. Student use of illicit drugs 

9. Student possession of weapons 

10. Physical abuse of teachers 

1 1 . Verbal abuse of teachers 
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The enumerated conditions were rated either as serious, moderate, minor or non-problematic. 
The eleven school climate questions used in the analysis were identically worded on the school 
questionnaires for all three grade levels. 

Data Analysis Procedures 

The data for the present analysis were obtained from the public-use data tape provided 
by NAEP, allowing outside researchers to make use of the NAEP database. In addition to the 
raw data from the three school questionnaires, SPSS-X control files were also made available 
by NAEP support personnel at ETS, which facilitated the creation of system files containing 
variable definitions, variable and value labels, and missing value statements. 

For the purposes of analyzing the questions of policy change and school climate, two 
composite scaled scores of these constructs were created, using the individual questionnaire 
Items as input. For the questions related to policy change, each item was receded to produce 
a dichotomous (0,1) scoreable item. This was achieved by collapsing the two "yes" categories 
(change by school policy and change by state or district policy) into one general category 
indicating change in policy as opposed to no change effected. Thus if a school indicated 
change either as a result of school policy or as mandated by the ctate or school district, its 
response was categorized as a "1", whereas schools responding with no change were coded 
as "0". The responses to the eight questions comprising the school policy change variable 
were then summed to yield an index of policy change in the school, ranging on a scale from 
0 (no change) to 8 (maximum change). 

A school climate composite was also created based on the responses to the eleven 
questions dealing with problems in the school. Each variable was measured on a four point 
scale, v/ith values of 4 for serious, 3 for moderate, 2 for minor and 1 for not a problem. The 
composite was then derived by summing the scores for the eleven variables and dividing by 
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eleven, yielding an index of school climate for each school. This index was based on the 
same scale as the original vari&bies, ranging from 1 (no problems) to 4 (serious problems). 

Using the aeated composites of policy change and school climate, as well as the 
individual variables comprising the composites as the outcomes of interest, the purpose of the 
analysis was to investigate along which dimensions the various outcomes could be 
differentiated. A set of five grouping variables was selected as the basis of determining what 
patterns, if any, existed in the data. The five variables selected were as follows: 

1. SCSTOC = Size and Type of Community 

1. Extreme Rural - Schools in rural areas with a high proportion of 
adults employed in farming. 

2. Low Metropolitan - Schools in urban areas with populations greater 
than 200,000 where a high proportion of adults were unemployed or on 
welfare. 

3. High Metropolitan - Schools in uriDan areri with populations greater 
than 200,000 where a high proportion of adults were in professional or 
managerial positions. 

4. Main Big City - Schools located in urban areas with populations 
greater than 200,000 but not falling into the categories of High or Low 
Metro. 

5. Urban Fringe - Schools located outside the city limits Oi urban areas 
with populations greater than 200.000. but not categorized as High or Low 
Metro. 

6. Medium City - Schools located in urban areas with populations 
between 25.000 and 200.000. but not classifiable as urban fringe areas. 
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7. Small Place • Schools located in areas with populations of less than 
25,000, not located in the urban areas of big cities and not falling into the 
category of Extreme Rural. 
This is a variable provided by Westat, Inc., the NAEP survey subcontractor, which was 
responsible for Implementing the 1988 sampling design. Participating schools were placed 
Into the above categories on the basis of type of community, size of population, and an 
occupational profile of residents provided by school principals. 

2. SORSHPT = Orshansky Percentile 

The Orshansky index is a variable based on information provided by Quality Education 
Data, Inc. (QED), which annually maintains and updates school level information. The index 
is an indicator of relative wealth specifying what percent of school-age children fall below the 
poverty line in a particular district. For the purposes of this analysis, the index was grouped 
as follows (in percents): 

1. 0-2 Little or no poverty 

2. 3-10 Low degree of poverty 

3. 1 1 -20 Moderate degree of poverty 

4. > 20 High degree of poverty 

3. SENROLL = Student Enrollment Code 

1. 1-99 students 

2. 100-299 ' 

3. 300-499 " 

4. 500-749 " 

5. > 749 " 

The student enrollment code was also obtained from QED and was a measure of school size. 
The code initially consisted of seven categories, but was receded with the effect of labeling 
schools with more than 749 students as "large" schools. 
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4. PPCWHT = Percent White 

This variable was obtained from the Principal's Questionnaire (PO) administered by 
Westat, Inc. prior to conducting the assessment. As a measure of percent of student ethnicity 
in each school, the variable was subsequently receded and broken down as follows (in 
percents): 

1. 0-49 Majority minority 

2. 50-89 Integrated 

3. > 89 Predominantly white 

5. SPLUNCH = Percent In School Lunch Program 

This variable was obtained from an ETS d*- ived estimate indicating the percent of 
students in a given school receiving a subsidized lunch program. The estimate was derived 
by dividing the number of students receiving a subsidized lunch program (as indicated on the 
school questionnaire) by the number of students In the school (from the best estimate of 
student enrollment from either the PQ or QED). The variable was used to yield an additional 
Indicator of school socioeconomic status and was broken down as follows for analysis (in 
percents): 

1. 0-12 Low 

2. 13-25 Low/Moderate 

3. 28-50 Moderate/High 

4. > 50 High 

Weighting and Estimation Procedures 

All reported results in this study are based on weighted responses to the various 
questionnaire items. NAEP uses a complex sampling scheme, rather than simple random 
sampling, in conducting the national assessment. Under this stratified multistage probability 
sampling design, certain subgroups (e.g., minorities) are sampled at higher rates in order to 
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ensure adequate precision in the estimation of sut)population characteristics. Each school 
sampled, thus, is assigned a sampling weight to reflect the differential representation of various 
subgroups in the sample. The use of weighted responses in analyzing these data adjusts for 
the differential sampling rates to produce approximately unbiased sample estimates of the 
various subpopulation parameters. (For further information on the details of NAEP's sampling 
scheme, please refer to the National Assessment of Educational Progress 1988 Public-Use 
Data Tapes User Guide. Version 1.0 .) 

Although the use of weighted responses in analyzing the school questionnaire data 
resuJis in sample estimates of population parameters such as means, proportiors and 
regression coefficients which are approximately unbiased, standard statistical tests based on 
a naive use of these weighted data are not meaningful. This is in part because the sampling 
weights are designed to sum to the estimated total population size. Consequently, statistical 
tests whose degrees of freedom are based on the sum of the weights are almost guaranteed 
to be significant. To correct for this phenomenon, new scaled weights were created to 
approximate the sample n's actually used and mal<e it possible to perform traditional statistical 
tests more appropriately. In addition, any statistical tests should also be based on sampling 
variability estimates appropriate to the sampling design one employs. NAEP uses a statistical 
procedure refen-ed to as jacl<l<nlfing to produce accurate standard errors, which corrects for the 
underestimation of the true sampling variability that would be produced by conventional 
variance estimates based on the assumption of a simple random sampling design. The 
analyses in this study do not employ the use of jackknifed standard errors, and thus are 
subject to misinterpretation if undue importance is attached to the statistical significance of the 
results. 

For the purposes of this analysis a design effect expressing the average ratio of a 
hypothetical jackknifed variance estimate to the actual variance obtained (assuming simple 
random sampling) was conservatively defined at 1 .5 (as recommended by Eugene Johnson 
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of ETS). This design effect was then used to define an effective sample size, expressed as 
actual sample size/design effect (1.5), which was then used for the analyses at the three 
grade levels. The standard errors reported in this study are the products of employing the 
design effect and the new scaled weights and thus approximately account for the sampling 
design. For further details on this procedure, please refer to the NAEP User Guide . Section 
8.3, Procedures used by NAEP to estimate sampling variability. 

For the above reasons, all statistical results reported and discussed should be 
interpreted with some degree of caution. Moreover, the performance of a large group of 
statistical tests on the same data can result in some statistically significant findings occurring 
merely by chance. Means and percentages (weighted to yield population estimates) of the 
various policy change and school climate variables will be presented in a descriptive manner 
with an eye to detecting consistent patterns in the data among the various groupings. Formal 
tests of statistical significance (One-way Analysis of Variance and Chi Square) have also been 
perfomied and reported for these data. One should be aware when reviewing these data that 
some results, while indicating statistical significance, may not be large enough to have any 
practical significar^ce for policy considerations. 

Results/Findings 

A. Policy Changes by Grade Level 

The policy change composite score, based on a scale from 0 (no change) to 8 
(maximum change) increased by grade level from a low mean score of 2.92 for grade four to 
a score of 3.11 for grade eight up to 3.55 for grade twelve. These average composite scores 
indicate the greatest amount of policy change since the 1984-1985 school year occurring at the 
high school level. 

When looked at by the five grouping variables, various patterns emerge which remain 
fairly consistent across grade levels (see Table 1). The size and type of community measure 



ERIC 



12 



11 

(SCSTOC) showed the highest policy composite score for the category Low Metro for grades 
four (4.0''0 and twelve (4.18). while for grade eight Low Metro was ranked second with a mean 
of 4.05 next to Main Big City (4.30). Toe differences among the various levels of SCSTOC 
were statistically significant as indicated in the table at gtados four and eight, but not, however 
at grade twelve. (Statistical results from the One-way Analysis of Variance performed on these 
data are based on an overall F test among the levels of the grouping variables). Since the 
results fpr Low Metro and Main Big City are insignificantly different from each other at each 
grade level, these results show schools located in disadvantaged urban areas and in main big 
cities registering the greatest amount of policy change for grades four and eight. 

As far as the other grouping variables are concemed, there is a consistent pattern of 
results across the three grade levels. For the Or£hansl<y percentile (SORS'-iPT), there exists 
a moderate to strong relationship between level of poverty and policy change, with the greatest 
amount of change occurring In the high poverty schools. Results at all three grade levels were 
statistically significant at the .05 level. Similarly for school enrollment (SENROLL), the largest 
schools exhibit the highest degree of policy change across grade level. The relationship 
between school size and policy change Is lairly consistent for grades four and eight, but grade 
twelve shows less vaiiaoiiity among the five enrollment categories. Accordingly, results at 
grades four and eight were statistically significarit, but not at grade twelve. 

For the /ariable Pleasuring percent students receiving a subsidized lunch program 
(SPLUNCH), the amount of policy change is highest in the category where more than 50 
percent of the students receive free lunch. Similarly, the amount of change is least In the 
category 0-12% for all three grades. A fairly consistent relationship exists between the two 
variables at all grade levels, although the amount of variation is less at grade twelve. Results 
were statistically significant only at grade eight, where the variability among the various levels 
of SPLUNCH is the greatest. 
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Table 1 



School Policy Change h.eans*** 
By Grade Level Broken Down by Groupings 

Grade 4 Grade 8 Grade 12 



SCSTOC 














cxtfGme nurai 


o An* 


(.38) 


o no* 

o.Oo 


/ OC\ 

(.36) 


A A^ 

3.26 


/ A^\ 

(.27) 


Low Metro 


A AC 

4.06 


(.55) 


4.05 


(.42) 


4.18 


(.P2) 


nign MGiro 


O.Of 


(.49) 


2.26 


/ OC\ 

(.36) 


2.73 


(.-r3) 


Main big uity 


0.95 


(.51) 


4.30 


(.45) 


3.60 


/ PA\ 

(.53) 


Urban Fringe 


2.45 


(.41) 


3.25 


(.29) 


3.57 


(.45) 


Medium City 


2.93 


(.32) 


2.86 


(.47) 


3.98 


(.35) 


small Place 


2.41 


(.25) 


2.76 


(.18) 


3.63 


(.18) 


C/^DCLJDT 

oUnolir 1 














u • ^ 


O CO* 


/ OCX 

(.25) 


A A4 * 
2.21 


/ OA\ 

(.20) 


3.07 


/ A4 \ 

(.21) 


3 - 10 


2.44 


(.26) 


3.01 


(.21) 


3.09 


(.22) 


11 -20 


3.39 


(.32) 


3.83 


(.25) 


4.18 


(.24) 


>20 


3.84 


(.31) 


J.87 


(.28) 


4.21 


(.29) 


oLIMnULL 














1 -99 


1.23* 


(.24) 


2.11* 


(.44) 


3.55 


(.45) 


100 - 299 


3.16 


(.24) 


2.50 


(.20) 


3.54 


(.21) 


300 - 499 


2.64 


(.26) 


2.92 


(.22) 


3S3 


(.30) 




Q AA 


(.37) 


o.ob 


/ OQ\ 

(.28) 


3.03 


{.cO) 


>749 


3.56 


(.52) 


4.14 


(.30) 


3.96 


(.26) 


SPLUNCH 














0 - 12 


2.72 


(.36) 


3.00* 


(.23) 


3.53 


(.22) 


13-25 


3.25 


(.38) 


3.08 


(.22) 


3.60 


(.27) 


26 - 50 


2.98 


(.30) 


3.45 


(.30) 


3.69 


(.23) 


>50 


3.60 


(.37) 


4.16 


(.41) 


3.76 


(.81) 


PPCWHT 














0 -49 


3.80* 


(.37) 


4.20* 


(.28) 


4.11 


(.54) 


50-89 


3.58 


(.31) 


2.67 


(.27) 


3.78 


(.24) 


>89 


2.51 


(.19) 


2.70 


(.15) 


3.35 


(.15) 


TOTALS: 


2.92 


(.15) 


3.11 


(.13) 


3.55 


(.12) 



* Differences among levels of grouping variable significant at 0.05 level 
Standard errors in parentheses 



Means are weighted to yield population estimates 
Bold indicates largest mean within grouping 
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Finally, the percunt white grouping variable (PPCWHT) also shows a clear relationship 
between the percent of white students in a school and the amount of policy change. The 
highest degree of change is found in those schools with the lowest percentage of white 
students (majority minority), while the predominantly white schools show considerably less 
policy chan^^e across the three grade levels. Results for this grouping variable were statistically 
significant at grades four and eight, but not at grade twelve. 

In summary, the results for the policy change composite score brol^en down on the 
f ve grouping variables suggest that the highest degree of reported change might occur in 
'je, urban disadvantaged schools with majority minority populations at all three grade levels. 
At grade eight, differences among the level of all five grouping variables were statistically 
significant at 0.05 according to the One-way Analysis of Variance test. (See Figure 1 for a 
graphical display). Conversely, reported change is apt to be lass at small, predominantly white, 
less disadvantaged schools.* 

B. School Climate by Grade Level 

School climate, as previously mentioned, represents an average composite of eleven 
school-leval variables dealing with matter^ ranging from absenteeism to physical abuse of 
teachers. Based on a scale from 1 (no problem) to 4 (serious), the average score ranged 
from a low of 1.27 at grade four, increasing to 1.44 at grade eight and to 1.69 at grade twelve, 
repeating a similar pattern to the policy change composite (see Table 2). 

When brol^en down by the individual grouping variables, similar patterns to policy change 
also emerge.** In temns of size and type of community, the highest average degree of reported 
problems occurs in the category Low Metro across all g ade levels, with the category Main Big 

'Although a multivariate analysis S' as discriminant function analysis would be 
needed to confirm this. 

"Similarly, grade eight showed statistically significant differences among the levels of 
the five grouping variables at the 0,05 level. 
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City raiiked second for grades four and eight, and third for grade twelve. Results for SCSTOC 
were statistically significant at .05 for grades four and eight, but not at grade twelve. 

In term of the Orshansky percentile, the data show that as levels of poverty increase, 
a corresponding increase in the school climate score occurs as well. Although the relationship 
is not entirely linear, especially at grades eight and twri^'^^, the higher school climate scores 
(more serious problams) are associated with higher levels of school poverty, while lower scores 
occur with little or no poverty. Results at all three grade levels were found to be statistically 
significant at the .00 level. 

The school size variable. SENROi.!^ shows a strong corespondence with school 
climate, espedally at grades eight and twelve where the results were statistically significant. 
At all grade levels, the highest school climate scores are assodated with the largest schools, 
while the lowest scores generally occur with the smallest schools (with the exception of grade 
four where the lowest two categories of enrollment are slightly reversed by .01). 

The grouping variable measuring percent students receiving a subsidized lunch program 
shows a moderate relationship with school dimate at grades fOur and eight, and surprisingly, 
a slightly negative relationship at grade twelve. At grades four and eight, the highest school 
dimate scores occur in schools where more than 50% of the students receive free lunch, while 
at grade twelve the situation is reversed. The lowest category of school lunch comprising 0 
to 12% of the students has the highest level of school problems, although only .07 of a 
difference score separates the highest and lowest categories. Results at all three grade levels 
were found to be statistically significant. 

Finally, the pccent white grouping variable displays a moderate to strong relationship 
with school climate. The highest scores occur in schools with majority minority populations at 
all three grade levels while lower scores are associated with schools with predominantly white 
populations. Although the relationship is slightly reversed at the upper two categories for grade 
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Table 2 



School Climate Means*** 
By Grade Level Broken Down by Groupings 



Grade 4 Grade 8 Grade 12 



SCSTOC 












Extreme Rural 


1.30* 


(.05)** 


1.38* 


(.06) 


1.57 (.06) 


Low Metro 


1.49 


(.06) 


1.77 


(.09) 


1.90 (.19) 


High Metro 


1.20 


(.04) 


1.27 


(.05) 


1.71 (.08) 


Main 3ig City 


1.34 


(.06) 


1.54 


(.08) 


1.76 (.15) 


Urban Fringe 


1.24 


(.04) 


1.45 


(.08) 


1.87 (.09) 


Medium City 


1.21 


(.04) 


1.39 


i'.07) 


1.75 (.12) 


Small Place 


1.27 


(.03) 


1.41 


(.04) 


l!66 (.04) 


SORSHPT 












0-2 


1.21 


(.02) 


1.24 


(.03) 


1.49* (.05) 


3-10 


1.20 


(.03) 


1.47 


(.05) 


1.79 (.06) 


11 - 20 


1.34 


(.03) 


1.60 


(.05) 


1.84 (06) 


>20 


1.44 


(.04) 


1.56 


(.07) 


1.71 (.06) 


SENROLL 












1 - 99 


1.24 


(.05) 


1.19* 


(.05) 


1.39* (.10) 


100 - 299 


1.23 


(.03) 


1.35 


(.04) 


1 67 ( OB) 


300 - 499 


1.28 


(.03) 


1.38 


(.04) 


1.65 (.06) 


500 - 749 


1.32 


(.04) 


1.60 


(.05) 


1.66 (.04) 


>f 49 


1.36 


(.06) 


1.67 


(.07) 


1.88 (.06) 


SPLUNCH 












0 - 12 


1.17* 


(.03) 


1.45* 


(.04) 


1.81* (.04) 


13 - 25 


1.29 


(.04) 


1.40 


(.05) 


1.77 (.05) 


26 - 50 


1.30 


(.04) 


1.59 


(.06) 


1.57 (.07) 


>50 


1.42 


(.04) 


1.62 


(.08) 


1.74 (.23) 


PPCWHT 












0 - 49 


1.40* 


(.04) 


1.63* 


(.07) 


1.93* (.14) 


50 - 89 


1.29 


(.03} 


1.45 


(.06) 


1.61 (.07) 


>89 


1.22 


(.02) 


1.36 


(.03) 


1.71 (.03) 


TOTALS: 


1.27 


(.02) 


1.44 


(.02) 


1.69 (.03) 



* Difference^: among levels of grouping variable significant at 0.05 level 

Standard errors in parentheses 
*** Means are weighted to yield population estimates 
^ Bold indicates largest mean within grouping 
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twelve, the relationship is fairly linear at grades four and eight. 
PPCWHT were found to be statistically significant at p=.05. 
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Results at all grade levels for 



C. Policy Change: Individual Factors 

In attempting to account for which of the individual policy change variibles account for 
the variation In the composite scale score, a number of patterns clearly emerge (see Table 3). 
For grades four and eight, the establishment of new consistently enforced codes of student 
conduct had the highest ranking (63.1 and 67.7% respectively) of those schools reporting 
changes. For grade tvelve, the establishment of a stricter attendance policy had the highest 
ranking (72.6%), while conduct came in a close second, tied with establishing grade 
requirements for athletics or extra-curricular activities. The overall patterns as indicated by the 
ranking of the individual Variables are rather similar by grade. As indicated in the table, the 
three above mentioned variable.? of conduct, attendance and athletics exhibit the highest degree 
of change, while lengthening the school year and implementing a performance-based 
compensation system for teachers assume less importance at all grade levels. 



Table 3 



Policy Change Percentages* By Grade Level 
(With Ranks) 



Common Policy 
Changes 



Grade 4 



Rank 



Grade 8 Rank 



Grade 12 Rank 



School Day 34.1(3.3)** 5 30.2(2.9) 6 

School Year 18.3(2.7) 8 16.0(2.3) 8 

Homework 32.1(3.2) 6 30.3(2.9) 5 

Competency 36.7(3.4) 4 36.0(3.1) 4 

Conduc* 63.1(3.3) 1 67.7(3.0) 1 

Attendance 46.1(3.5) 2 51.3(3.2) 3 

Athletics 37.3(3.4) 3 60.4(3.1) 2 

Compensation 26.5(3.1) 7 21.7(2.6) 7 



40.0(3.5) 4 

16.7(2.7) 8 

26.5(3.2) 6 

37.7(3.5) 5 

69.5(3.3) 2.5 

72.6(3.2) 1 

69.5(3.3) 2.5 

23.4(3.0) 7 



* Percentages are weighted to yield population estimates. 



Standard errors are in parentheses. 



+ Bold figures indicate highest percentage within grade. 
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D. School Climate: Individual Factors 

Analyzing school climate by the individual variables comprising the composite scale 
score leads to several revealing Insights as shown by Table 4. First of all. problems tend to 
get more serious as one moves from grades fOur through grade eight up to grade twelve fOr 
all variables with the exception of physical conflicts among students (slight decrease at grade 
twelve). Tardiness and absenteeism rank as the two most serious problems across all grade 
levels. Student use of alcohol is also a more serious problem at grade twelve. (Interpretations 
of the relative severity of these problems should take into account the fact that the scale runs 
from 1 to 4 with a score of 4 characterizing a serious problem). 

Table 4 



School Climate Means* by Grade Level 
(With Ranks) 



Schnol Climate 
Indicators 



Grade 4 Rank 



Rank 


Grade 12 


Rank 


1 


2.31(.05) 


1 


2 


2.28(.06) 


2 


9 


1.67(.05) 


5 


3 


1.53(.05) 


8 


5 


1.63(.05) 


6 


4 


1.60(.04) 


7 


7 


2.16(.06) 


3 


8 


1.82(.05) 


4 


10 


1.12(.03) 


10 


11 


1.05(.02) 


11 


6 


1.46(.04) 


9 



Tardiness 
Absenteeism 
Cutting Classes 
Physical Conflicts 
Robbery/Theft 
Vandalism 
Alcohol Use 
Drug Use 
Weapons 
Phystoal Abuse 
Verbal Abuse 



1.72(.05)** 1 
1.65(.05) 2 
1.04(.01) 9.5 
1.60(.04) 3 
1.28(.03) 
1.33(.04) 
1.09(.03) 
1.07(.02) 
1.04(.01) 9.5 
1.02(.01) 11 
1.21 (.03) 6 



1.91 (.05) 
1.90(.05) 
1.25(.03) 
1.65(.04) 
1.44(.04) 
1.47(.04) 
1.40(.05) 
1.27(.03) 
1.11 (.02) 
1.03(.01) 
1.42(.04) 



* Means are weighted to yield population estimates 

Standard errors are in parentheses 
•f Bold indicates highest means within grade 

The ranking of the individual variable means is almost identical for grades four and 

eight, while grade twelve shows a somewhat different pattern in that use of alcohol and drugs 

as well as cutting classes are more pronounced at that grade level. 
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Conclusions/Summary 

The results reported in this study point to a number of summary conclusions. First of 
all, policy reforms seem to be linked to a number of societal and school-level factors. Based 
on the available data from the three school questionnaires, policy change appears to occur 
more often In large urban schools with large numbers of disadvantaged minority students. 
Correspondingly, school climate as measured by a number of school-related problems exhibits 
similar trends as indicated by the data. Matters of concern to school personnel, such as 
absenteeism and tardiness, tend to be more prevalent along the same dimensions as policy 
refomi; that is, school-related problems are associated more with bigger city schools containing 
large minority and less advantaged populations- For further details on multiple comparison test 
procedures concerning specific levels of the grouping variables on the policy change and school 
climate composite scores, please refer to the appendix. 

No attempt is made here, however, to draw any strong causal inferences f<om these 
data. Along with the problems inherent in using self-reported survey data, these data remain 
nonexperimental and there is thus little basis for validating any causal linkage between the 
variables of policy change and school climate. The data do support the hypothesis, 
nonetheless, that policy change aid school-related problems tend to co-occur in the same 
settings and suggest therefore that schools more affected by problems are making attempts 
to redress them through policy efforts. The opposing, seemingly implausible conclusion one 
car draw from these results is that policy change efforts are contributing to an increase in 
school-related problems! 

Additional effort needs to be devoted to studying possible interactions between the 
constructs of policy change and school climate. For example, is there any relationship between 
establishing a strict attendance policy in a school and the problem of absenteeism? Or, do 
schools which have reported establishing new consistently enforced codes of student conduct 
also tend to report fewer instances of physical conflicts among siudents or use of alcohol or 
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drugs? This study provides a basis for developing further initiatives in investigating the 
usefulness of school questionnaire data to Inform policy refonn efforts in our nation's schools. 



o 

ERIC 
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